Introduction

In the previous notebboks, we try to classify song based on audio signal using differents methods. Now, we will explore meta-data provided with the dataset. For now we will use only the reduced one with only 8k song but if the result is good, another notebbok will be setup with the full dataset.

With the dataset we have 2 metadata:

• The first one is provided by Echo Nest. It provides different features like acousticness ou speechness which evaluate song with specific factors. Unfortunately, echonest has been bought by Spotify and the API is not availabel for songs out of Spotify. By luck, it exists Traktomizer which do nearly the same.

• The second one can be computed easily with librosa. It's a group of feature extracted from the audio like the cqt or the sftf of the song.

In addition of the missing access to echonest API, we have plenty of missing data with this dataset so we will only use the extracted features.

Loading datas

import os
import glob
import pandas as pd
import numpy as np
import librosa
import random
import time 
import pickle
import queue
import threading

from scipy import signal
from scipy.spatial.distance import cdist

from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelBinarizer
from sklearn.ensemble import ExtraTreesClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegressionCV
from sklearn.svm import LinearSVC
from sklearn.svm import SVC
from sklearn.preprocessing import MinMaxScaler
from sklearn.decomposition import PCA

from keras.models import Sequential
from keras.layers import Dense, Flatten, Dropout
from keras.models import Model
from keras import backend as K
from keras.utils.generic_utils import get_custom_objects
from keras.layers import Activation

import IPython.display as ipd

import matplotlib
import matplotlib.pyplot as plt

np.random.seed(42)

%matplotlib inline

pd.set_option('max_info_columns', 999)

def swish(x):
    return x*K.sigmoid(x)

get_custom_objects().update({'custom_activation': swish})

C:\python36\envs\machine_learning\lib\site-packages\h5py\__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Using TensorFlow backend.

Tracks

tracks = pd.read_csv("F:/Nicolas/DNUPycharmProjects/machine_learning/audio/FMA/preprocessed_meta/tracks.csv", 
                     header=[0, 1], 
                     skipinitialspace=True, 
                     index_col=0, 
                    encoding = "ISO-8859-1")
tracks.head()

	album										...	track
	comments	date_created	date_released	engineer	favorites	id	information	listens	producer	tags	...	information	interest	language_code	license	listens	lyricist	number	publisher	tags	title
track_id
2	0	2008-11-26 01:44:45	2009-01-05 00:00:00	NaN	4	1	<p></p>	6073	NaN	[]	...	NaN	4656	en	Attribution-NonCommercial-ShareAlike 3.0 Inter...	1293	NaN	3	NaN	[]	Food
5	0	2008-11-26 01:44:45	2009-01-05 00:00:00	NaN	4	1	<p></p>	6073	NaN	[]	...	NaN	1933	en	Attribution-NonCommercial-ShareAlike 3.0 Inter...	1151	NaN	6	NaN	[]	This World
10	0	2008-11-26 01:45:08	2008-02-06 00:00:00	NaN	4	6	NaN	47632	NaN	[]	...	NaN	54881	en	Attribution-NonCommercial-NoDerivatives (aka M...	50135	NaN	1	NaN	[]	Freeway
140	1	2008-11-26 01:49:59	2007-05-22 00:00:00	NaN	1	61	<p>Alec K. Redfearn &amp; The Eyesores: Ellen ...	1300	Alec K. Refearn, Rob Pemberton	[]	...	NaN	1593	en	Attribution-Noncommercial-No Derivative Works ...	1299	NaN	2	NaN	[]	Queen Of The Wires
141	0	2008-11-26 01:49:57	2009-01-16 00:00:00	NaN	1	60	<p>A full ensamble of strings, drums, electron...	1304	NaN	[]	...	NaN	839	en	Attribution-Noncommercial-No Derivative Works ...	725	NaN	4	NaN	[]	Ohio

5 rows × 52 columns

On this dataset, we only need the top class for all 8k songs, this is for our y label

tracks = tracks[[ ("track", "genre_top") ]]

Echonest data

echonest = pd.read_csv("F:/Nicolas/DNUPycharmProjects/machine_learning/audio/FMA/preprocessed_meta/echonest.csv", index_col=0)
echonest.head()

	audio_features	audio_features.1	audio_features.2	audio_features.3	audio_features.4	audio_features.5	audio_features.6	audio_features.7	temporal_features	temporal_features.1	...	temporal_features.214	temporal_features.215	temporal_features.216	temporal_features.217	temporal_features.218	temporal_features.219	temporal_features.220	temporal_features.221	temporal_features.222	temporal_features.223
track_id	acousticness	danceability	energy	instrumentalness	liveness	speechiness	tempo	valence	0.000000	1.000000	...	214.000000	215.000000	216.000000	217.00000	218.000000	219.00000	220.00000	221.00000	222.000000	223.000000
2	0.4166752327	0.6758939853	0.6344762684	0.010628068300000001	0.17764657120000002	0.1593100648	165.922	0.576660988	0.877233	0.588911	...	-1.992303	6.805694	0.233070	0.19288	0.027455	0.06408	3.67696	3.61288	13.316690	262.929749
5	0.0435668989	0.7455658702	0.7014699916	0.0006967989999999999	0.37314331240000004	0.12459534189999999	100.26	0.6216612236	0.548093	0.720192	...	-2.288358	11.527109	0.256821	0.23782	0.060122	0.06014	5.92649	5.86635	16.013849	356.755737
10	0.9516699648	0.6581786543	0.9245251615000001	0.9654270154000001	0.1154738842	0.032985219100000004	111.562	0.9635898919	0.311404	0.711402	...	-3.662988	21.508228	0.283352	0.26707	0.125704	0.08082	8.41401	8.33319	21.317064	483.403809
140	0.3763124975	0.7340790229	0.2656847734	0.6695811237	0.08599512220000001	0.0390682262	107.952	0.6099912728	0.331811	0.312733	...	-0.934696	-0.260981	0.322232	0.27798	0.136747	0.07533	9.86272	9.78739	21.981621	562.229431

5 rows × 232 columns

echonest.info()

<class 'pandas.core.frame.DataFrame'>
Index: 1295 entries, track_id to 124186
Data columns (total 232 columns):
audio_features           1295 non-null object
audio_features.1         1295 non-null object
audio_features.2         1295 non-null object
audio_features.3         1295 non-null object
audio_features.4         1295 non-null object
audio_features.5         1295 non-null object
audio_features.6         1295 non-null object
audio_features.7         1295 non-null object
temporal_features        1295 non-null float64
temporal_features.1      1295 non-null float64
temporal_features.2      1295 non-null float64
temporal_features.3      1295 non-null float64
temporal_features.4      1295 non-null float64
temporal_features.5      1295 non-null float64
temporal_features.6      1295 non-null float64
temporal_features.7      1295 non-null float64
temporal_features.8      1295 non-null float64
temporal_features.9      1295 non-null float64
temporal_features.10     1295 non-null float64
temporal_features.11     1295 non-null float64
temporal_features.12     1295 non-null float64
temporal_features.13     1295 non-null float64
temporal_features.14     1295 non-null float64
temporal_features.15     1295 non-null float64
temporal_features.16     1295 non-null float64
temporal_features.17     1295 non-null float64
temporal_features.18     1295 non-null float64
temporal_features.19     1295 non-null float64
temporal_features.20     1295 non-null float64
temporal_features.21     1295 non-null float64
temporal_features.22     1295 non-null float64
temporal_features.23     1295 non-null float64
temporal_features.24     1295 non-null float64
temporal_features.25     1295 non-null float64
temporal_features.26     1295 non-null float64
temporal_features.27     1295 non-null float64
temporal_features.28     1295 non-null float64
temporal_features.29     1295 non-null float64
temporal_features.30     1295 non-null float64
temporal_features.31     1295 non-null float64
temporal_features.32     1295 non-null float64
temporal_features.33     1295 non-null float64
temporal_features.34     1295 non-null float64
temporal_features.35     1295 non-null float64
temporal_features.36     1295 non-null float64
temporal_features.37     1295 non-null float64
temporal_features.38     1295 non-null float64
temporal_features.39     1295 non-null float64
temporal_features.40     1295 non-null float64
temporal_features.41     1295 non-null float64
temporal_features.42     1295 non-null float64
temporal_features.43     1295 non-null float64
temporal_features.44     1295 non-null float64
temporal_features.45     1295 non-null float64
temporal_features.46     1295 non-null float64
temporal_features.47     1295 non-null float64
temporal_features.48     1295 non-null float64
temporal_features.49     1295 non-null float64
temporal_features.50     1295 non-null float64
temporal_features.51     1295 non-null float64
temporal_features.52     1295 non-null float64
temporal_features.53     1295 non-null float64
temporal_features.54     1295 non-null float64
temporal_features.55     1295 non-null float64
temporal_features.56     1295 non-null float64
temporal_features.57     1295 non-null float64
temporal_features.58     1295 non-null float64
temporal_features.59     1295 non-null float64
temporal_features.60     1295 non-null float64
temporal_features.61     1295 non-null float64
temporal_features.62     1295 non-null float64
temporal_features.63     1295 non-null float64
temporal_features.64     1295 non-null float64
temporal_features.65     1295 non-null float64
temporal_features.66     1295 non-null float64
temporal_features.67     1295 non-null float64
temporal_features.68     1295 non-null float64
temporal_features.69     1295 non-null float64
temporal_features.70     1295 non-null float64
temporal_features.71     1295 non-null float64
temporal_features.72     1295 non-null float64
temporal_features.73     1295 non-null float64
temporal_features.74     1295 non-null float64
temporal_features.75     1295 non-null float64
temporal_features.76     1295 non-null float64
temporal_features.77     1295 non-null float64
temporal_features.78     1295 non-null float64
temporal_features.79     1295 non-null float64
temporal_features.80     1295 non-null float64
temporal_features.81     1295 non-null float64
temporal_features.82     1295 non-null float64
temporal_features.83     1295 non-null float64
temporal_features.84     1295 non-null float64
temporal_features.85     1295 non-null float64
temporal_features.86     1295 non-null float64
temporal_features.87     1295 non-null float64
temporal_features.88     1295 non-null float64
temporal_features.89     1295 non-null float64
temporal_features.90     1295 non-null float64
temporal_features.91     1295 non-null float64
temporal_features.92     1295 non-null float64
temporal_features.93     1295 non-null float64
temporal_features.94     1295 non-null float64
temporal_features.95     1295 non-null float64
temporal_features.96     1295 non-null float64
temporal_features.97     1295 non-null float64
temporal_features.98     1295 non-null float64
temporal_features.99     1295 non-null float64
temporal_features.100    1295 non-null float64
temporal_features.101    1295 non-null float64
temporal_features.102    1295 non-null float64
temporal_features.103    1295 non-null float64
temporal_features.104    1295 non-null float64
temporal_features.105    1295 non-null float64
temporal_features.106    1295 non-null float64
temporal_features.107    1295 non-null float64
temporal_features.108    1295 non-null float64
temporal_features.109    1295 non-null float64
temporal_features.110    1295 non-null float64
temporal_features.111    1295 non-null float64
temporal_features.112    1295 non-null float64
temporal_features.113    1295 non-null float64
temporal_features.114    1295 non-null float64
temporal_features.115    1295 non-null float64
temporal_features.116    1295 non-null float64
temporal_features.117    1295 non-null float64
temporal_features.118    1295 non-null float64
temporal_features.119    1295 non-null float64
temporal_features.120    1295 non-null float64
temporal_features.121    1295 non-null float64
temporal_features.122    1295 non-null float64
temporal_features.123    1295 non-null float64
temporal_features.124    1295 non-null float64
temporal_features.125    1295 non-null float64
temporal_features.126    1295 non-null float64
temporal_features.127    1295 non-null float64
temporal_features.128    1295 non-null float64
temporal_features.129    1295 non-null float64
temporal_features.130    1295 non-null float64
temporal_features.131    1295 non-null float64
temporal_features.132    1295 non-null float64
temporal_features.133    1295 non-null float64
temporal_features.134    1295 non-null float64
temporal_features.135    1295 non-null float64
temporal_features.136    1295 non-null float64
temporal_features.137    1295 non-null float64
temporal_features.138    1295 non-null float64
temporal_features.139    1295 non-null float64
temporal_features.140    1295 non-null float64
temporal_features.141    1295 non-null float64
temporal_features.142    1295 non-null float64
temporal_features.143    1295 non-null float64
temporal_features.144    1295 non-null float64
temporal_features.145    1295 non-null float64
temporal_features.146    1295 non-null float64
temporal_features.147    1295 non-null float64
temporal_features.148    1295 non-null float64
temporal_features.149    1295 non-null float64
temporal_features.150    1295 non-null float64
temporal_features.151    1295 non-null float64
temporal_features.152    1295 non-null float64
temporal_features.153    1295 non-null float64
temporal_features.154    1295 non-null float64
temporal_features.155    1295 non-null float64
temporal_features.156    1295 non-null float64
temporal_features.157    1295 non-null float64
temporal_features.158    1295 non-null float64
temporal_features.159    1295 non-null float64
temporal_features.160    1295 non-null float64
temporal_features.161    1295 non-null float64
temporal_features.162    1295 non-null float64
temporal_features.163    1295 non-null float64
temporal_features.164    1295 non-null float64
temporal_features.165    1295 non-null float64
temporal_features.166    1295 non-null float64
temporal_features.167    1295 non-null float64
temporal_features.168    1295 non-null float64
temporal_features.169    1295 non-null float64
temporal_features.170    1295 non-null float64
temporal_features.171    1295 non-null float64
temporal_features.172    1295 non-null float64
temporal_features.173    1295 non-null float64
temporal_features.174    1295 non-null float64
temporal_features.175    1295 non-null float64
temporal_features.176    1295 non-null float64
temporal_features.177    1295 non-null float64
temporal_features.178    1295 non-null float64
temporal_features.179    1295 non-null float64
temporal_features.180    1295 non-null float64
temporal_features.181    1295 non-null float64
temporal_features.182    1295 non-null float64
temporal_features.183    1295 non-null float64
temporal_features.184    1295 non-null float64
temporal_features.185    1295 non-null float64
temporal_features.186    1295 non-null float64
temporal_features.187    1295 non-null float64
temporal_features.188    1295 non-null float64
temporal_features.189    1295 non-null float64
temporal_features.190    1295 non-null float64
temporal_features.191    1295 non-null float64
temporal_features.192    1295 non-null float64
temporal_features.193    1295 non-null float64
temporal_features.194    1295 non-null float64
temporal_features.195    1295 non-null float64
temporal_features.196    1295 non-null float64
temporal_features.197    1295 non-null float64
temporal_features.198    1295 non-null float64
temporal_features.199    1295 non-null float64
temporal_features.200    1295 non-null float64
temporal_features.201    1295 non-null float64
temporal_features.202    1295 non-null float64
temporal_features.203    1295 non-null float64
temporal_features.204    1295 non-null float64
temporal_features.205    1295 non-null float64
temporal_features.206    1295 non-null float64
temporal_features.207    1295 non-null float64
temporal_features.208    1295 non-null float64
temporal_features.209    1295 non-null float64
temporal_features.210    1295 non-null float64
temporal_features.211    1295 non-null float64
temporal_features.212    1295 non-null float64
temporal_features.213    1295 non-null float64
temporal_features.214    1295 non-null float64
temporal_features.215    1295 non-null float64
temporal_features.216    1295 non-null float64
temporal_features.217    1295 non-null float64
temporal_features.218    1295 non-null float64
temporal_features.219    1295 non-null float64
temporal_features.220    1295 non-null float64
temporal_features.221    1295 non-null float64
temporal_features.222    1295 non-null float64
temporal_features.223    1295 non-null float64
dtypes: float64(224), object(8)
memory usage: 2.3+ MB

As mentionned previously, on 8k songs, we have only 1295 audio with echonest analysis. That's why we will ignore it

Other features

features = pd.read_csv("F:/Nicolas/DNUPycharmProjects/machine_learning/audio/FMA/preprocessed_meta/features.csv", header=[0, 1, 2], skipinitialspace=True, index_col=0)
features.head()

feature	chroma_cens										...	tonnetz			zcr
statistics	kurtosis										...	std			kurtosis	max	mean	median	min	skew	std
number	01	02	03	04	05	06	07	08	09	10	...	04	05	06	01	01	01	01	01	01	01
track_id
2	7.180653	5.230309	0.249321	1.347620	1.482478	0.531371	1.481593	2.691455	0.866868	1.341231	...	0.054125	0.012226	0.012111	5.758890	0.459473	0.085629	0.071289	0.000000	2.089872	0.061448
5	0.527563	-0.077654	-0.279610	0.685883	1.937570	0.880839	-0.923192	-0.927232	0.666617	1.038546	...	0.040730	0.012691	0.014759	6.808415	0.375000	0.053114	0.041504	0.000000	2.193303	0.044861
10	3.702245	-0.291193	2.196742	-0.234449	1.367364	0.998411	1.770694	1.604566	0.521217	1.982386	...	0.074358	0.017952	0.013921	21.434212	0.452148	0.077515	0.071777	0.000000	3.542325	0.040800
140	0.533579	-0.623885	-1.086205	-1.081079	-0.765151	-0.072282	-0.882913	-0.582376	-0.884749	-0.645214	...	0.157683	0.028070	0.025946	11.052547	0.379395	0.052379	0.036621	0.001953	3.143968	0.057712
141	0.172898	-0.284804	-1.169662	-1.062855	-0.706868	-0.708281	-0.204884	0.023624	-0.642770	-0.786291	...	0.145994	0.024342	0.032111	32.994659	0.415527	0.040267	0.034668	0.002930	4.204097	0.028665

5 rows × 518 columns

features.info()

<class 'pandas.core.frame.DataFrame'>
Int64Index: 7994 entries, 2 to 155066
Data columns (total 518 columns):
(chroma_cens, kurtosis, 01)           7994 non-null float64
(chroma_cens, kurtosis, 02)           7994 non-null float64
(chroma_cens, kurtosis, 03)           7994 non-null float64
(chroma_cens, kurtosis, 04)           7994 non-null float64
(chroma_cens, kurtosis, 05)           7994 non-null float64
(chroma_cens, kurtosis, 06)           7994 non-null float64
(chroma_cens, kurtosis, 07)           7994 non-null float64
(chroma_cens, kurtosis, 08)           7994 non-null float64
(chroma_cens, kurtosis, 09)           7994 non-null float64
(chroma_cens, kurtosis, 10)           7994 non-null float64
(chroma_cens, kurtosis, 11)           7994 non-null float64
(chroma_cens, kurtosis, 12)           7994 non-null float64
(chroma_cens, max, 01)                7994 non-null float64
(chroma_cens, max, 02)                7994 non-null float64
(chroma_cens, max, 03)                7994 non-null float64
(chroma_cens, max, 04)                7994 non-null float64
(chroma_cens, max, 05)                7994 non-null float64
(chroma_cens, max, 06)                7994 non-null float64
(chroma_cens, max, 07)                7994 non-null float64
(chroma_cens, max, 08)                7994 non-null float64
(chroma_cens, max, 09)                7994 non-null float64
(chroma_cens, max, 10)                7994 non-null float64
(chroma_cens, max, 11)                7994 non-null float64
(chroma_cens, max, 12)                7994 non-null float64
(chroma_cens, mean, 01)               7994 non-null float64
(chroma_cens, mean, 02)               7994 non-null float64
(chroma_cens, mean, 03)               7994 non-null float64
(chroma_cens, mean, 04)               7994 non-null float64
(chroma_cens, mean, 05)               7994 non-null float64
(chroma_cens, mean, 06)               7994 non-null float64
(chroma_cens, mean, 07)               7994 non-null float64
(chroma_cens, mean, 08)               7994 non-null float64
(chroma_cens, mean, 09)               7994 non-null float64
(chroma_cens, mean, 10)               7994 non-null float64
(chroma_cens, mean, 11)               7994 non-null float64
(chroma_cens, mean, 12)               7994 non-null float64
(chroma_cens, median, 01)             7994 non-null float64
(chroma_cens, median, 02)             7994 non-null float64
(chroma_cens, median, 03)             7994 non-null float64
(chroma_cens, median, 04)             7994 non-null float64
(chroma_cens, median, 05)             7994 non-null float64
(chroma_cens, median, 06)             7994 non-null float64
(chroma_cens, median, 07)             7994 non-null float64
(chroma_cens, median, 08)             7994 non-null float64
(chroma_cens, median, 09)             7994 non-null float64
(chroma_cens, median, 10)             7994 non-null float64
(chroma_cens, median, 11)             7994 non-null float64
(chroma_cens, median, 12)             7994 non-null float64
(chroma_cens, min, 01)                7994 non-null float64
(chroma_cens, min, 02)                7994 non-null float64
(chroma_cens, min, 03)                7994 non-null float64
(chroma_cens, min, 04)                7994 non-null float64
(chroma_cens, min, 05)                7994 non-null float64
(chroma_cens, min, 06)                7994 non-null float64
(chroma_cens, min, 07)                7994 non-null float64
(chroma_cens, min, 08)                7994 non-null float64
(chroma_cens, min, 09)                7994 non-null float64
(chroma_cens, min, 10)                7994 non-null float64
(chroma_cens, min, 11)                7994 non-null float64
(chroma_cens, min, 12)                7994 non-null float64
(chroma_cens, skew, 01)               7994 non-null float64
(chroma_cens, skew, 02)               7994 non-null float64
(chroma_cens, skew, 03)               7994 non-null float64
(chroma_cens, skew, 04)               7994 non-null float64
(chroma_cens, skew, 05)               7994 non-null float64
(chroma_cens, skew, 06)               7994 non-null float64
(chroma_cens, skew, 07)               7994 non-null float64
(chroma_cens, skew, 08)               7994 non-null float64
(chroma_cens, skew, 09)               7994 non-null float64
(chroma_cens, skew, 10)               7994 non-null float64
(chroma_cens, skew, 11)               7994 non-null float64
(chroma_cens, skew, 12)               7994 non-null float64
(chroma_cens, std, 01)                7994 non-null float64
(chroma_cens, std, 02)                7994 non-null float64
(chroma_cens, std, 03)                7994 non-null float64
(chroma_cens, std, 04)                7994 non-null float64
(chroma_cens, std, 05)                7994 non-null float64
(chroma_cens, std, 06)                7994 non-null float64
(chroma_cens, std, 07)                7994 non-null float64
(chroma_cens, std, 08)                7994 non-null float64
(chroma_cens, std, 09)                7994 non-null float64
(chroma_cens, std, 10)                7994 non-null float64
(chroma_cens, std, 11)                7994 non-null float64
(chroma_cens, std, 12)                7994 non-null float64
(chroma_cqt, kurtosis, 01)            7994 non-null float64
(chroma_cqt, kurtosis, 02)            7994 non-null float64
(chroma_cqt, kurtosis, 03)            7994 non-null float64
(chroma_cqt, kurtosis, 04)            7994 non-null float64
(chroma_cqt, kurtosis, 05)            7994 non-null float64
(chroma_cqt, kurtosis, 06)            7994 non-null float64
(chroma_cqt, kurtosis, 07)            7994 non-null float64
(chroma_cqt, kurtosis, 08)            7994 non-null float64
(chroma_cqt, kurtosis, 09)            7994 non-null float64
(chroma_cqt, kurtosis, 10)            7994 non-null float64
(chroma_cqt, kurtosis, 11)            7994 non-null float64
(chroma_cqt, kurtosis, 12)            7994 non-null float64
(chroma_cqt, max, 01)                 7994 non-null float64
(chroma_cqt, max, 02)                 7994 non-null float64
(chroma_cqt, max, 03)                 7994 non-null float64
(chroma_cqt, max, 04)                 7994 non-null float64
(chroma_cqt, max, 05)                 7994 non-null float64
(chroma_cqt, max, 06)                 7994 non-null float64
(chroma_cqt, max, 07)                 7994 non-null float64
(chroma_cqt, max, 08)                 7994 non-null float64
(chroma_cqt, max, 09)                 7994 non-null float64
(chroma_cqt, max, 10)                 7994 non-null float64
(chroma_cqt, max, 11)                 7994 non-null float64
(chroma_cqt, max, 12)                 7994 non-null float64
(chroma_cqt, mean, 01)                7994 non-null float64
(chroma_cqt, mean, 02)                7994 non-null float64
(chroma_cqt, mean, 03)                7994 non-null float64
(chroma_cqt, mean, 04)                7994 non-null float64
(chroma_cqt, mean, 05)                7994 non-null float64
(chroma_cqt, mean, 06)                7994 non-null float64
(chroma_cqt, mean, 07)                7994 non-null float64
(chroma_cqt, mean, 08)                7994 non-null float64
(chroma_cqt, mean, 09)                7994 non-null float64
(chroma_cqt, mean, 10)                7994 non-null float64
(chroma_cqt, mean, 11)                7994 non-null float64
(chroma_cqt, mean, 12)                7994 non-null float64
(chroma_cqt, median, 01)              7994 non-null float64
(chroma_cqt, median, 02)              7994 non-null float64
(chroma_cqt, median, 03)              7994 non-null float64
(chroma_cqt, median, 04)              7994 non-null float64
(chroma_cqt, median, 05)              7994 non-null float64
(chroma_cqt, median, 06)              7994 non-null float64
(chroma_cqt, median, 07)              7994 non-null float64
(chroma_cqt, median, 08)              7994 non-null float64
(chroma_cqt, median, 09)              7994 non-null float64
(chroma_cqt, median, 10)              7994 non-null float64
(chroma_cqt, median, 11)              7994 non-null float64
(chroma_cqt, median, 12)              7994 non-null float64
(chroma_cqt, min, 01)                 7994 non-null float64
(chroma_cqt, min, 02)                 7994 non-null float64
(chroma_cqt, min, 03)                 7994 non-null float64
(chroma_cqt, min, 04)                 7994 non-null float64
(chroma_cqt, min, 05)                 7994 non-null float64
(chroma_cqt, min, 06)                 7994 non-null float64
(chroma_cqt, min, 07)                 7994 non-null float64
(chroma_cqt, min, 08)                 7994 non-null float64
(chroma_cqt, min, 09)                 7994 non-null float64
(chroma_cqt, min, 10)                 7994 non-null float64
(chroma_cqt, min, 11)                 7994 non-null float64
(chroma_cqt, min, 12)                 7994 non-null float64
(chroma_cqt, skew, 01)                7994 non-null float64
(chroma_cqt, skew, 02)                7994 non-null float64
(chroma_cqt, skew, 03)                7994 non-null float64
(chroma_cqt, skew, 04)                7994 non-null float64
(chroma_cqt, skew, 05)                7994 non-null float64
(chroma_cqt, skew, 06)                7994 non-null float64
(chroma_cqt, skew, 07)                7994 non-null float64
(chroma_cqt, skew, 08)                7994 non-null float64
(chroma_cqt, skew, 09)                7994 non-null float64
(chroma_cqt, skew, 10)                7994 non-null float64
(chroma_cqt, skew, 11)                7994 non-null float64
(chroma_cqt, skew, 12)                7994 non-null float64
(chroma_cqt, std, 01)                 7994 non-null float64
(chroma_cqt, std, 02)                 7994 non-null float64
(chroma_cqt, std, 03)                 7994 non-null float64
(chroma_cqt, std, 04)                 7994 non-null float64
(chroma_cqt, std, 05)                 7994 non-null float64
(chroma_cqt, std, 06)                 7994 non-null float64
(chroma_cqt, std, 07)                 7994 non-null float64
(chroma_cqt, std, 08)                 7994 non-null float64
(chroma_cqt, std, 09)                 7994 non-null float64
(chroma_cqt, std, 10)                 7994 non-null float64
(chroma_cqt, std, 11)                 7994 non-null float64
(chroma_cqt, std, 12)                 7994 non-null float64
(chroma_stft, kurtosis, 01)           7994 non-null float64
(chroma_stft, kurtosis, 02)           7994 non-null float64
(chroma_stft, kurtosis, 03)           7994 non-null float64
(chroma_stft, kurtosis, 04)           7994 non-null float64
(chroma_stft, kurtosis, 05)           7994 non-null float64
(chroma_stft, kurtosis, 06)           7994 non-null float64
(chroma_stft, kurtosis, 07)           7994 non-null float64
(chroma_stft, kurtosis, 08)           7994 non-null float64
(chroma_stft, kurtosis, 09)           7994 non-null float64
(chroma_stft, kurtosis, 10)           7994 non-null float64
(chroma_stft, kurtosis, 11)           7994 non-null float64
(chroma_stft, kurtosis, 12)           7994 non-null float64
(chroma_stft, max, 01)                7994 non-null float64
(chroma_stft, max, 02)                7994 non-null float64
(chroma_stft, max, 03)                7994 non-null float64
(chroma_stft, max, 04)                7994 non-null float64
(chroma_stft, max, 05)                7994 non-null float64
(chroma_stft, max, 06)                7994 non-null float64
(chroma_stft, max, 07)                7994 non-null float64
(chroma_stft, max, 08)                7994 non-null float64
(chroma_stft, max, 09)                7994 non-null float64
(chroma_stft, max, 10)                7994 non-null float64
(chroma_stft, max, 11)                7994 non-null float64
(chroma_stft, max, 12)                7994 non-null float64
(chroma_stft, mean, 01)               7994 non-null float64
(chroma_stft, mean, 02)               7994 non-null float64
(chroma_stft, mean, 03)               7994 non-null float64
(chroma_stft, mean, 04)               7994 non-null float64
(chroma_stft, mean, 05)               7994 non-null float64
(chroma_stft, mean, 06)               7994 non-null float64
(chroma_stft, mean, 07)               7994 non-null float64
(chroma_stft, mean, 08)               7994 non-null float64
(chroma_stft, mean, 09)               7994 non-null float64
(chroma_stft, mean, 10)               7994 non-null float64
(chroma_stft, mean, 11)               7994 non-null float64
(chroma_stft, mean, 12)               7994 non-null float64
(chroma_stft, median, 01)             7994 non-null float64
(chroma_stft, median, 02)             7994 non-null float64
(chroma_stft, median, 03)             7994 non-null float64
(chroma_stft, median, 04)             7994 non-null float64
(chroma_stft, median, 05)             7994 non-null float64
(chroma_stft, median, 06)             7994 non-null float64
(chroma_stft, median, 07)             7994 non-null float64
(chroma_stft, median, 08)             7994 non-null float64
(chroma_stft, median, 09)             7994 non-null float64
(chroma_stft, median, 10)             7994 non-null float64
(chroma_stft, median, 11)             7994 non-null float64
(chroma_stft, median, 12)             7994 non-null float64
(chroma_stft, min, 01)                7994 non-null float64
(chroma_stft, min, 02)                7994 non-null float64
(chroma_stft, min, 03)                7994 non-null float64
(chroma_stft, min, 04)                7994 non-null float64
(chroma_stft, min, 05)                7994 non-null float64
(chroma_stft, min, 06)                7994 non-null float64
(chroma_stft, min, 07)                7994 non-null float64
(chroma_stft, min, 08)                7994 non-null float64
(chroma_stft, min, 09)                7994 non-null float64
(chroma_stft, min, 10)                7994 non-null float64
(chroma_stft, min, 11)                7994 non-null float64
(chroma_stft, min, 12)                7994 non-null float64
(chroma_stft, skew, 01)               7994 non-null float64
(chroma_stft, skew, 02)               7994 non-null float64
(chroma_stft, skew, 03)               7994 non-null float64
(chroma_stft, skew, 04)               7994 non-null float64
(chroma_stft, skew, 05)               7994 non-null float64
(chroma_stft, skew, 06)               7994 non-null float64
(chroma_stft, skew, 07)               7994 non-null float64
(chroma_stft, skew, 08)               7994 non-null float64
(chroma_stft, skew, 09)               7994 non-null float64
(chroma_stft, skew, 10)               7994 non-null float64
(chroma_stft, skew, 11)               7994 non-null float64
(chroma_stft, skew, 12)               7994 non-null float64
(chroma_stft, std, 01)                7994 non-null float64
(chroma_stft, std, 02)                7994 non-null float64
(chroma_stft, std, 03)                7994 non-null float64
(chroma_stft, std, 04)                7994 non-null float64
(chroma_stft, std, 05)                7994 non-null float64
(chroma_stft, std, 06)                7994 non-null float64
(chroma_stft, std, 07)                7994 non-null float64
(chroma_stft, std, 08)                7994 non-null float64
(chroma_stft, std, 09)                7994 non-null float64
(chroma_stft, std, 10)                7994 non-null float64
(chroma_stft, std, 11)                7994 non-null float64
(chroma_stft, std, 12)                7994 non-null float64
(mfcc, kurtosis, 01)                  7994 non-null float64
(mfcc, kurtosis, 02)                  7994 non-null float64
(mfcc, kurtosis, 03)                  7994 non-null float64
(mfcc, kurtosis, 04)                  7994 non-null float64
(mfcc, kurtosis, 05)                  7994 non-null float64
(mfcc, kurtosis, 06)                  7994 non-null float64
(mfcc, kurtosis, 07)                  7994 non-null float64
(mfcc, kurtosis, 08)                  7994 non-null float64
(mfcc, kurtosis, 09)                  7994 non-null float64
(mfcc, kurtosis, 10)                  7994 non-null float64
(mfcc, kurtosis, 11)                  7994 non-null float64
(mfcc, kurtosis, 12)                  7994 non-null float64
(mfcc, kurtosis, 13)                  7994 non-null float64
(mfcc, kurtosis, 14)                  7994 non-null float64
(mfcc, kurtosis, 15)                  7994 non-null float64
(mfcc, kurtosis, 16)                  7994 non-null float64
(mfcc, kurtosis, 17)                  7994 non-null float64
(mfcc, kurtosis, 18)                  7994 non-null float64
(mfcc, kurtosis, 19)                  7994 non-null float64
(mfcc, kurtosis, 20)                  7994 non-null float64
(mfcc, max, 01)                       7994 non-null float64
(mfcc, max, 02)                       7994 non-null float64
(mfcc, max, 03)                       7994 non-null float64
(mfcc, max, 04)                       7994 non-null float64
(mfcc, max, 05)                       7994 non-null float64
(mfcc, max, 06)                       7994 non-null float64
(mfcc, max, 07)                       7994 non-null float64
(mfcc, max, 08)                       7994 non-null float64
(mfcc, max, 09)                       7994 non-null float64
(mfcc, max, 10)                       7994 non-null float64
(mfcc, max, 11)                       7994 non-null float64
(mfcc, max, 12)                       7994 non-null float64
(mfcc, max, 13)                       7994 non-null float64
(mfcc, max, 14)                       7994 non-null float64
(mfcc, max, 15)                       7994 non-null float64
(mfcc, max, 16)                       7994 non-null float64
(mfcc, max, 17)                       7994 non-null float64
(mfcc, max, 18)                       7994 non-null float64
(mfcc, max, 19)                       7994 non-null float64
(mfcc, max, 20)                       7994 non-null float64
(mfcc, mean, 01)                      7994 non-null float64
(mfcc, mean, 02)                      7994 non-null float64
(mfcc, mean, 03)                      7994 non-null float64
(mfcc, mean, 04)                      7994 non-null float64
(mfcc, mean, 05)                      7994 non-null float64
(mfcc, mean, 06)                      7994 non-null float64
(mfcc, mean, 07)                      7994 non-null float64
(mfcc, mean, 08)                      7994 non-null float64
(mfcc, mean, 09)                      7994 non-null float64
(mfcc, mean, 10)                      7994 non-null float64
(mfcc, mean, 11)                      7994 non-null float64
(mfcc, mean, 12)                      7994 non-null float64
(mfcc, mean, 13)                      7994 non-null float64
(mfcc, mean, 14)                      7994 non-null float64
(mfcc, mean, 15)                      7994 non-null float64
(mfcc, mean, 16)                      7994 non-null float64
(mfcc, mean, 17)                      7994 non-null float64
(mfcc, mean, 18)                      7994 non-null float64
(mfcc, mean, 19)                      7994 non-null float64
(mfcc, mean, 20)                      7994 non-null float64
(mfcc, median, 01)                    7994 non-null float64
(mfcc, median, 02)                    7994 non-null float64
(mfcc, median, 03)                    7994 non-null float64
(mfcc, median, 04)                    7994 non-null float64
(mfcc, median, 05)                    7994 non-null float64
(mfcc, median, 06)                    7994 non-null float64
(mfcc, median, 07)                    7994 non-null float64
(mfcc, median, 08)                    7994 non-null float64
(mfcc, median, 09)                    7994 non-null float64
(mfcc, median, 10)                    7994 non-null float64
(mfcc, median, 11)                    7994 non-null float64
(mfcc, median, 12)                    7994 non-null float64
(mfcc, median, 13)                    7994 non-null float64
(mfcc, median, 14)                    7994 non-null float64
(mfcc, median, 15)                    7994 non-null float64
(mfcc, median, 16)                    7994 non-null float64
(mfcc, median, 17)                    7994 non-null float64
(mfcc, median, 18)                    7994 non-null float64
(mfcc, median, 19)                    7994 non-null float64
(mfcc, median, 20)                    7994 non-null float64
(mfcc, min, 01)                       7994 non-null float64
(mfcc, min, 02)                       7994 non-null float64
(mfcc, min, 03)                       7994 non-null float64
(mfcc, min, 04)                       7994 non-null float64
(mfcc, min, 05)                       7994 non-null float64
(mfcc, min, 06)                       7994 non-null float64
(mfcc, min, 07)                       7994 non-null float64
(mfcc, min, 08)                       7994 non-null float64
(mfcc, min, 09)                       7994 non-null float64
(mfcc, min, 10)                       7994 non-null float64
(mfcc, min, 11)                       7994 non-null float64
(mfcc, min, 12)                       7994 non-null float64
(mfcc, min, 13)                       7994 non-null float64
(mfcc, min, 14)                       7994 non-null float64
(mfcc, min, 15)                       7994 non-null float64
(mfcc, min, 16)                       7994 non-null float64
(mfcc, min, 17)                       7994 non-null float64
(mfcc, min, 18)                       7994 non-null float64
(mfcc, min, 19)                       7994 non-null float64
(mfcc, min, 20)                       7994 non-null float64
(mfcc, skew, 01)                      7994 non-null float64
(mfcc, skew, 02)                      7994 non-null float64
(mfcc, skew, 03)                      7994 non-null float64
(mfcc, skew, 04)                      7994 non-null float64
(mfcc, skew, 05)                      7994 non-null float64
(mfcc, skew, 06)                      7994 non-null float64
(mfcc, skew, 07)                      7994 non-null float64
(mfcc, skew, 08)                      7994 non-null float64
(mfcc, skew, 09)                      7994 non-null float64
(mfcc, skew, 10)                      7994 non-null float64
(mfcc, skew, 11)                      7994 non-null float64
(mfcc, skew, 12)                      7994 non-null float64
(mfcc, skew, 13)                      7994 non-null float64
(mfcc, skew, 14)                      7994 non-null float64
(mfcc, skew, 15)                      7994 non-null float64
(mfcc, skew, 16)                      7994 non-null float64
(mfcc, skew, 17)                      7994 non-null float64
(mfcc, skew, 18)                      7994 non-null float64
(mfcc, skew, 19)                      7994 non-null float64
(mfcc, skew, 20)                      7994 non-null float64
(mfcc, std, 01)                       7994 non-null float64
(mfcc, std, 02)                       7994 non-null float64
(mfcc, std, 03)                       7994 non-null float64
(mfcc, std, 04)                       7994 non-null float64
(mfcc, std, 05)                       7994 non-null float64
(mfcc, std, 06)                       7994 non-null float64
(mfcc, std, 07)                       7994 non-null float64
(mfcc, std, 08)                       7994 non-null float64
(mfcc, std, 09)                       7994 non-null float64
(mfcc, std, 10)                       7994 non-null float64
(mfcc, std, 11)                       7994 non-null float64
(mfcc, std, 12)                       7994 non-null float64
(mfcc, std, 13)                       7994 non-null float64
(mfcc, std, 14)                       7994 non-null float64
(mfcc, std, 15)                       7994 non-null float64
(mfcc, std, 16)                       7994 non-null float64
(mfcc, std, 17)                       7994 non-null float64
(mfcc, std, 18)                       7994 non-null float64
(mfcc, std, 19)                       7994 non-null float64
(mfcc, std, 20)                       7994 non-null float64
(rmse, kurtosis, 01)                  7994 non-null float64
(rmse, max, 01)                       7994 non-null float64
(rmse, mean, 01)                      7994 non-null float64
(rmse, median, 01)                    7994 non-null float64
(rmse, min, 01)                       7994 non-null float64
(rmse, skew, 01)                      7994 non-null float64
(rmse, std, 01)                       7994 non-null float64
(spectral_bandwidth, kurtosis, 01)    7994 non-null float64
(spectral_bandwidth, max, 01)         7994 non-null float64
(spectral_bandwidth, mean, 01)        7994 non-null float64
(spectral_bandwidth, median, 01)      7994 non-null float64
(spectral_bandwidth, min, 01)         7994 non-null float64
(spectral_bandwidth, skew, 01)        7994 non-null float64
(spectral_bandwidth, std, 01)         7994 non-null float64
(spectral_centroid, kurtosis, 01)     7994 non-null float64
(spectral_centroid, max, 01)          7994 non-null float64
(spectral_centroid, mean, 01)         7994 non-null float64
(spectral_centroid, median, 01)       7994 non-null float64
(spectral_centroid, min, 01)          7994 non-null float64
(spectral_centroid, skew, 01)         7994 non-null float64
(spectral_centroid, std, 01)          7994 non-null float64
(spectral_contrast, kurtosis, 01)     7994 non-null float64
(spectral_contrast, kurtosis, 02)     7994 non-null float64
(spectral_contrast, kurtosis, 03)     7994 non-null float64
(spectral_contrast, kurtosis, 04)     7994 non-null float64
(spectral_contrast, kurtosis, 05)     7994 non-null float64
(spectral_contrast, kurtosis, 06)     7994 non-null float64
(spectral_contrast, kurtosis, 07)     7994 non-null float64
(spectral_contrast, max, 01)          7994 non-null float64
(spectral_contrast, max, 02)          7994 non-null float64
(spectral_contrast, max, 03)          7994 non-null float64
(spectral_contrast, max, 04)          7994 non-null float64
(spectral_contrast, max, 05)          7994 non-null float64
(spectral_contrast, max, 06)          7994 non-null float64
(spectral_contrast, max, 07)          7994 non-null float64
(spectral_contrast, mean, 01)         7994 non-null float64
(spectral_contrast, mean, 02)         7994 non-null float64
(spectral_contrast, mean, 03)         7994 non-null float64
(spectral_contrast, mean, 04)         7994 non-null float64
(spectral_contrast, mean, 05)         7994 non-null float64
(spectral_contrast, mean, 06)         7994 non-null float64
(spectral_contrast, mean, 07)         7994 non-null float64
(spectral_contrast, median, 01)       7994 non-null float64
(spectral_contrast, median, 02)       7994 non-null float64
(spectral_contrast, median, 03)       7994 non-null float64
(spectral_contrast, median, 04)       7994 non-null float64
(spectral_contrast, median, 05)       7994 non-null float64
(spectral_contrast, median, 06)       7994 non-null float64
(spectral_contrast, median, 07)       7994 non-null float64
(spectral_contrast, min, 01)          7994 non-null float64
(spectral_contrast, min, 02)          7994 non-null float64
(spectral_contrast, min, 03)          7994 non-null float64
(spectral_contrast, min, 04)          7994 non-null float64
(spectral_contrast, min, 05)          7994 non-null float64
(spectral_contrast, min, 06)          7994 non-null float64
(spectral_contrast, min, 07)          7994 non-null float64
(spectral_contrast, skew, 01)         7994 non-null float64
(spectral_contrast, skew, 02)         7994 non-null float64
(spectral_contrast, skew, 03)         7994 non-null float64
(spectral_contrast, skew, 04)         7994 non-null float64
(spectral_contrast, skew, 05)         7994 non-null float64
(spectral_contrast, skew, 06)         7994 non-null float64
(spectral_contrast, skew, 07)         7994 non-null float64
(spectral_contrast, std, 01)          7994 non-null float64
(spectral_contrast, std, 02)          7994 non-null float64
(spectral_contrast, std, 03)          7994 non-null float64
(spectral_contrast, std, 04)          7994 non-null float64
(spectral_contrast, std, 05)          7994 non-null float64
(spectral_contrast, std, 06)          7994 non-null float64
(spectral_contrast, std, 07)          7994 non-null float64
(spectral_rolloff, kurtosis, 01)      7994 non-null float64
(spectral_rolloff, max, 01)           7994 non-null float64
(spectral_rolloff, mean, 01)          7994 non-null float64
(spectral_rolloff, median, 01)        7994 non-null float64
(spectral_rolloff, min, 01)           7994 non-null float64
(spectral_rolloff, skew, 01)          7994 non-null float64
(spectral_rolloff, std, 01)           7994 non-null float64
(tonnetz, kurtosis, 01)               7994 non-null float64
(tonnetz, kurtosis, 02)               7994 non-null float64
(tonnetz, kurtosis, 03)               7994 non-null float64
(tonnetz, kurtosis, 04)               7994 non-null float64
(tonnetz, kurtosis, 05)               7994 non-null float64
(tonnetz, kurtosis, 06)               7994 non-null float64
(tonnetz, max, 01)                    7994 non-null float64
(tonnetz, max, 02)                    7994 non-null float64
(tonnetz, max, 03)                    7994 non-null float64
(tonnetz, max, 04)                    7994 non-null float64
(tonnetz, max, 05)                    7994 non-null float64
(tonnetz, max, 06)                    7994 non-null float64
(tonnetz, mean, 01)                   7994 non-null float64
(tonnetz, mean, 02)                   7994 non-null float64
(tonnetz, mean, 03)                   7994 non-null float64
(tonnetz, mean, 04)                   7994 non-null float64
(tonnetz, mean, 05)                   7994 non-null float64
(tonnetz, mean, 06)                   7994 non-null float64
(tonnetz, median, 01)                 7994 non-null float64
(tonnetz, median, 02)                 7994 non-null float64
(tonnetz, median, 03)                 7994 non-null float64
(tonnetz, median, 04)                 7994 non-null float64
(tonnetz, median, 05)                 7994 non-null float64
(tonnetz, median, 06)                 7994 non-null float64
(tonnetz, min, 01)                    7994 non-null float64
(tonnetz, min, 02)                    7994 non-null float64
(tonnetz, min, 03)                    7994 non-null float64
(tonnetz, min, 04)                    7994 non-null float64
(tonnetz, min, 05)                    7994 non-null float64
(tonnetz, min, 06)                    7994 non-null float64
(tonnetz, skew, 01)                   7994 non-null float64
(tonnetz, skew, 02)                   7994 non-null float64
(tonnetz, skew, 03)                   7994 non-null float64
(tonnetz, skew, 04)                   7994 non-null float64
(tonnetz, skew, 05)                   7994 non-null float64
(tonnetz, skew, 06)                   7994 non-null float64
(tonnetz, std, 01)                    7994 non-null float64
(tonnetz, std, 02)                    7994 non-null float64
(tonnetz, std, 03)                    7994 non-null float64
(tonnetz, std, 04)                    7994 non-null float64
(tonnetz, std, 05)                    7994 non-null float64
(tonnetz, std, 06)                    7994 non-null float64
(zcr, kurtosis, 01)                   7994 non-null float64
(zcr, max, 01)                        7994 non-null float64
(zcr, mean, 01)                       7994 non-null float64
(zcr, median, 01)                     7994 non-null float64
(zcr, min, 01)                        7994 non-null float64
(zcr, skew, 01)                       7994 non-null float64
(zcr, std, 01)                        7994 non-null float64
dtypes: float64(518)
memory usage: 31.7 MB

As we can see there is plenty of features. To see it more clerly, let's plot a boxplot for all group of features. This will help to see outliers

group = sorted(list(set([col[:2] for col in features])))

filtering

First we can see that we don't have all data with Echonest. As a result we cannot exploit it as is. we may use it for visualization (will be done later with R) to see it there is high correlation between features and classes. But for now, we will focus on the extracted features (things we can do with librosa for free)

for a in group:
    subset = features[a[0]][a[1]]
    plt.figure(figsize=(20, 8))
    boxplot = subset.boxplot()
    plt.title(a)
    plt.show()

Based on this, we can remove songs which seems to be outliers (there is only few of them = 16/7995). In a future notebook, we will try to improve scaling, cleaning but for now, the objective is only to see if it outperform our previous models

features = features[ features['chroma_cens']['kurtosis']['03'] < 150 ]
features = features[ features['chroma_cens']['kurtosis']['08'] < 150 ]
features = features[ features['chroma_cens']['kurtosis']['11'] < 150 ]

features = features[ features['chroma_cqt']['kurtosis']['03'] < 150 ]

features = features[ features['chroma_stft']['kurtosis']['01'] < 150 ]

features = features[ features['chroma_stft']['skew']['01'] > -25 ]

features = features[ features['mfcc']['kurtosis']['01'] < 100 ]
features = features[ features['mfcc']['kurtosis']['02'] < 100 ]

features = features[ features['rmse']['kurtosis']['01'] < 100 ]

features = features[ features['spectral_bandwidth']['kurtosis']['01'] < 400 ]

Preparation

tracks = tracks[tracks.index.isin(features.index)]

First let's scale feature. For now, we will apply MinMaxScaling for all features. We don't apply log transformation or Standard Scaling. This may improve performances but this is not the question yet.

X = MinMaxScaler().fit_transform(features)

index_song = features.index.values

Now, let's reduce dimensions as much as possible

pca = PCA(n_components=0.99)
new_X = pca.fit_transform(X)

X.shape

(7979, 518)

new_X.shape

(7979, 284)

PCA is quite efficient here, we reduce dimensions by 46% by losing only 1% of the variance. To finish let's encode and split the dataset to train/test

y = tracks[("track", "genre_top")].values

enc = LabelBinarizer()
y_enc = enc.fit_transform(y.reshape(-1, 1))

X_train, X_test, y_train, y_test, y_enc_train, y_enc_test = train_test_split(new_X, y, y_enc, test_size=0.20, random_state=42)

Models

We can now try some models

clf = SVC()
clf.fit(X_train, y_train)

print(clf.score(X_train, y_train))
print(clf.score(X_test, y_test))

0.5047783174056086
0.5025062656641605

clf = LinearSVC(multi_class = "crammer_singer")

clf.fit(X_train, y_train)

print(clf.score(X_train, y_train))
print(clf.score(X_test, y_test))

0.6763277455741814
0.5632832080200502

clf = LogisticRegressionCV(multi_class="multinomial")

clf.fit(X_train, y_train)

print(clf.score(X_train, y_train))
print(clf.score(X_test, y_test))

0.6293279022403259
0.5720551378446115

for depth in range(5, 12):
    clf = RandomForestClassifier(n_estimators = 50, max_depth=depth)
    clf.fit(X_train, y_train)
    print("\nDepth:", depth)
    print(clf.score(X_train, y_train))
    print(clf.score(X_test, y_test))

Depth: 5
0.5965846780510732
0.4780701754385965

Depth: 6
0.6428011906626978
0.46365914786967416

Depth: 7
0.7380542064859784
0.4630325814536341

Depth: 8
0.8030706564311453
0.4617794486215539

Depth: 9
0.8987936706877644
0.4692982456140351

Depth: 10
0.9454801817327275
0.4674185463659148

Depth: 11
0.9807300642331193
0.46365914786967416

for depth in range(5, 12):
    clf = ExtraTreesClassifier(n_estimators = 50, max_depth=depth)
    clf.fit(X_train, y_train)
    print("\nDepth:", depth)
    print(clf.score(X_train, y_train))
    print(clf.score(X_test, y_test))

Depth: 5
0.5903180322732258
0.44799498746867167

Depth: 6
0.6501644994516685
0.4492481203007519

Depth: 7
0.7336675544414852
0.4548872180451128

Depth: 8
0.8044806517311609
0.46365914786967416

Depth: 9
0.8654237819207269
0.4473684210526316

Depth: 10
0.9163402788657371
0.45802005012531327

Depth: 11
0.9461068463105123
0.4605263157894737

model = Sequential()
model.add(Dense(128, input_dim=X_train.shape[1], activation='custom_activation'))
model.add(Dropout(0.4))
model.add(Dense(32, activation='custom_activation'))
model.add(Dropout(0.2))
model.add(Dense(8, activation='softmax'))

model.summary()

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_1 (Dense)              (None, 128)               36480     
_________________________________________________________________
dropout_1 (Dropout)          (None, 128)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 32)                4128      
_________________________________________________________________
dropout_2 (Dropout)          (None, 32)                0         
_________________________________________________________________
dense_3 (Dense)              (None, 8)                 264       
=================================================================
Total params: 40,872
Trainable params: 40,872
Non-trainable params: 0
_________________________________________________________________

model.compile(loss='binary_crossentropy', 
              optimizer='Nadam', 
              metrics=["binary_crossentropy", "accuracy"])

history = model.fit(X_train, y_enc_train, 
                      epochs=25, 
                      batch_size=100, 
                      verbose=1, 
                      validation_data=(X_test, y_enc_test))

Train on 6383 samples, validate on 1596 samples
Epoch 1/25
6383/6383 [==============================] - 2s 289us/step - loss: 0.3415 - binary_crossentropy: 0.3415 - acc: 0.8766 - val_loss: 0.2933 - val_binary_crossentropy: 0.2933 - val_acc: 0.8839
Epoch 2/25
6383/6383 [==============================] - 0s 57us/step - loss: 0.2745 - binary_crossentropy: 0.2745 - acc: 0.8937 - val_loss: 0.2535 - val_binary_crossentropy: 0.2535 - val_acc: 0.9014
Epoch 3/25
6383/6383 [==============================] - 0s 54us/step - loss: 0.2484 - binary_crossentropy: 0.2484 - acc: 0.9028 - val_loss: 0.2402 - val_binary_crossentropy: 0.2402 - val_acc: 0.9052
Epoch 4/25
6383/6383 [==============================] - 0s 54us/step - loss: 0.2357 - binary_crossentropy: 0.2357 - acc: 0.9086 - val_loss: 0.2372 - val_binary_crossentropy: 0.2372 - val_acc: 0.9059
Epoch 5/25
6383/6383 [==============================] - 0s 54us/step - loss: 0.2277 - binary_crossentropy: 0.2277 - acc: 0.9122 - val_loss: 0.2356 - val_binary_crossentropy: 0.2356 - val_acc: 0.9066
Epoch 6/25
6383/6383 [==============================] - 0s 49us/step - loss: 0.2227 - binary_crossentropy: 0.2227 - acc: 0.9140 - val_loss: 0.2346 - val_binary_crossentropy: 0.2346 - val_acc: 0.9072
Epoch 7/25
6383/6383 [==============================] - 0s 50us/step - loss: 0.2182 - binary_crossentropy: 0.2182 - acc: 0.9155 - val_loss: 0.2342 - val_binary_crossentropy: 0.2342 - val_acc: 0.9080
Epoch 8/25
6383/6383 [==============================] - 0s 62us/step - loss: 0.2132 - binary_crossentropy: 0.2132 - acc: 0.9186 - val_loss: 0.2346 - val_binary_crossentropy: 0.2346 - val_acc: 0.9084
Epoch 9/25
6383/6383 [==============================] - 0s 53us/step - loss: 0.2111 - binary_crossentropy: 0.2111 - acc: 0.9197 - val_loss: 0.2344 - val_binary_crossentropy: 0.2344 - val_acc: 0.9084
Epoch 10/25
6383/6383 [==============================] - 0s 54us/step - loss: 0.2088 - binary_crossentropy: 0.2088 - acc: 0.9190 - val_loss: 0.2348 - val_binary_crossentropy: 0.2348 - val_acc: 0.9081
Epoch 11/25
6383/6383 [==============================] - 0s 51us/step - loss: 0.2064 - binary_crossentropy: 0.2064 - acc: 0.9203 - val_loss: 0.2345 - val_binary_crossentropy: 0.2345 - val_acc: 0.9095
Epoch 12/25
6383/6383 [==============================] - 0s 56us/step - loss: 0.2045 - binary_crossentropy: 0.2045 - acc: 0.9197 - val_loss: 0.2367 - val_binary_crossentropy: 0.2367 - val_acc: 0.9066
Epoch 13/25
6383/6383 [==============================] - 0s 55us/step - loss: 0.2033 - binary_crossentropy: 0.2033 - acc: 0.9219 - val_loss: 0.2367 - val_binary_crossentropy: 0.2367 - val_acc: 0.9073
Epoch 14/25
6383/6383 [==============================] - 0s 54us/step - loss: 0.2028 - binary_crossentropy: 0.2028 - acc: 0.9216 - val_loss: 0.2374 - val_binary_crossentropy: 0.2374 - val_acc: 0.9072
Epoch 15/25
6383/6383 [==============================] - 0s 56us/step - loss: 0.2001 - binary_crossentropy: 0.2001 - acc: 0.9223 - val_loss: 0.2372 - val_binary_crossentropy: 0.2372 - val_acc: 0.9063
Epoch 16/25
6383/6383 [==============================] - 0s 58us/step - loss: 0.1987 - binary_crossentropy: 0.1987 - acc: 0.9227 - val_loss: 0.2372 - val_binary_crossentropy: 0.2372 - val_acc: 0.9061
Epoch 17/25
6383/6383 [==============================] - 0s 55us/step - loss: 0.1958 - binary_crossentropy: 0.1958 - acc: 0.9234 - val_loss: 0.2386 - val_binary_crossentropy: 0.2386 - val_acc: 0.9065
Epoch 18/25
6383/6383 [==============================] - 0s 56us/step - loss: 0.1956 - binary_crossentropy: 0.1956 - acc: 0.9246 - val_loss: 0.2382 - val_binary_crossentropy: 0.2382 - val_acc: 0.9070
Epoch 19/25
6383/6383 [==============================] - 0s 55us/step - loss: 0.1940 - binary_crossentropy: 0.1940 - acc: 0.9248 - val_loss: 0.2388 - val_binary_crossentropy: 0.2388 - val_acc: 0.9069
Epoch 20/25
6383/6383 [==============================] - 0s 57us/step - loss: 0.1915 - binary_crossentropy: 0.1915 - acc: 0.9266 - val_loss: 0.2380 - val_binary_crossentropy: 0.2380 - val_acc: 0.9068
Epoch 21/25
6383/6383 [==============================] - 0s 52us/step - loss: 0.1930 - binary_crossentropy: 0.1930 - acc: 0.9251 - val_loss: 0.2385 - val_binary_crossentropy: 0.2385 - val_acc: 0.9068
Epoch 22/25
6383/6383 [==============================] - 0s 52us/step - loss: 0.1892 - binary_crossentropy: 0.1892 - acc: 0.9268 - val_loss: 0.2392 - val_binary_crossentropy: 0.2392 - val_acc: 0.9063
Epoch 23/25
6383/6383 [==============================] - 0s 52us/step - loss: 0.1880 - binary_crossentropy: 0.1880 - acc: 0.9266 - val_loss: 0.2397 - val_binary_crossentropy: 0.2397 - val_acc: 0.9066
Epoch 24/25
6383/6383 [==============================] - 0s 52us/step - loss: 0.1878 - binary_crossentropy: 0.1878 - acc: 0.9263 - val_loss: 0.2404 - val_binary_crossentropy: 0.2404 - val_acc: 0.9055
Epoch 25/25
6383/6383 [==============================] - 0s 51us/step - loss: 0.1864 - binary_crossentropy: 0.1864 - acc: 0.9268 - val_loss: 0.2406 - val_binary_crossentropy: 0.2406 - val_acc: 0.9068

I've done some triles with differents activation fonction at fixed topology 128 + D04 + 32 + D02 + 8S. For all of them I look at the sammlest test loss. Results are the following one :

• RELU = 0.2621395363470068
• ELU = 0.23712751105016933
• SWISH = 0.2321398657814303
• SELU = 0.2378648814178051
• tanh = 0.23728603737097337
• hard_sigmoid = 0.23761723707791857
• sigmoid = 0.23684624532111606
• linear = 0.24031202196327964

Results are quite good now with more than 90% accuracy of accucary. This is a good points because those features are easy to extract. We will try on a future notebbok to try it with more songs using the full dataset. In addition, having more datas will allow us to keep more dimensions without increasing overfitting. To finish this notebbok, let's check what are closest songs based on those scaled features

Recommander

To do so, we will compute for a given song, all distances with other songs and take the closest one. As all values are scaled, we can use euclidean metric. Let's see the result.

def get_audio(index):
    song_str = "{0:06d}".format(index)
    folder = song_str[:3]

    base_song_path = "F:/Nicolas/DNUPycharmProjects/machine_learning/audio/FMA/fma_small/"+folder+"/"+song_str+".mp3"
    y, sr = librosa.load(base_song_path, sr=None, mono = True)

    if sr != 22050:
        y = librosa.resample(y, sr, 22050)
        sr = 22050
    print("returning song {}".format(song_str))
    return y[:500000]

def create_playlist(index):
    vector = [new_X[index, :]]
    dist = cdist(new_X, vector)
    close_index = np.argsort(dist.flatten())[1]
    index_base_song = index_song[index]
    index_close_song = index_song[close_index]
    
    y_orig = get_audio(index_base_song)
    y_closest = get_audio(index_close_song)
    
    return y_orig, y_closest

y1, y2 = create_playlist(42)

returning song 000707
returning song 000705

ipd.Audio(y1, rate=22050)

ipd.Audio(y2, rate=22050)

We can hear that both audio have the same energy but they are still differents. Nevertheless it's still similar.

y1, y2 = create_playlist(31)

returning song 000621
returning song 054665

ipd.Audio(y1, rate=22050)

ipd.Audio(y2, rate=22050)

This prediction is better, we have in both case an acoustic song. One has more echo but there are correct. We have also to keep in mind that we have access to ONLY 8K audio samples of 30s. With more audio, we can find closest ones.

y1, y2 = create_playlist(1)

returning song 000005
returning song 119578

ipd.Audio(y1, rate=22050)

ipd.Audio(y2, rate=22050)

The last audio looks very differents but the second song change after 15s to another one more similar. This can be strange for the listener but at the end they are also similar.

Conclusion

In this notebook, we used meta data easy to extract from audio in order to classify song with their genre and also to do a recommander. This is not perfect but we may imagine extracting those features every 10s and then use RNN on a input of 60 timesteps x 584 features. This may help having better prediction if the audio change during time. Based on the result we have here, this is more likely what we have to do. This will be done after exploring dataset more in detail with R. The idea of this exploration is to find if we should convert to log some features for example.